Color misregistration reducer

ABSTRACT

A color shift reducing apparatus includes: a color shift area detecting unit for detecting a color shift area on the basis of input image data and image data delayed through a memory; a color shift degree calculating unit for calculating data indicating the degree of color shift on the basis of a color shift detection signal, the input image data, and the image data delayed by one field through the memory; an image data estimating unit for estimating image data on the basis of the color shift detection signal, the input image data, and the image data supplied from the memory; and a color shift reducing unit for forming a color shift reduced image on the basis of the image data supplied from the memory, estimated image data, and the color shift degree data.

TECHNICAL FIELD

The present invention relates to a color shift reducing apparatus forreducing color shift in color video signals derived from afield-sequential imaging device.

BACKGROUND ART

In general, an electronic endoscope includes imaging means such as animaging device for applying illumination light of R (red), G (green),and B (blue) with various wavelengths to a subject in a time seriesmanner and then photoelectrically converting light reflected by thesubject into video signals using a solid-state image sensor such as acharge coupled device (CCD). This type of imaging device has thefollowing disadvantage: When the subject moves fast, three colorcomponent signals obtained in time series are shifted. In other words,color shift occurs.

To reduce color shift, color shift reducing apparatuses for receivingcolor video signals from a field-sequential imaging device and reducingcolor shift of the color video signals have been proposed.

The color shift reducing apparatuses include a color shift reducingapparatus for applying a mean color component of the whole of an imagewith color shift to a color shift part, thus correcting the color shift,and a color shift reducing apparatus for forming a correction colorusing a brightness component and color components of an image and thenapplying the correction color to a color shift part, thus correcting thecolor shift as disclosed in Japanese Unexamined Patent ApplicationPublication No. 3-270392.

Japanese Unexamined Patent Application Publication No. 6-319694discloses an apparatus for extracting the highest pixel frequency valuefrom a color distribution histogram and assigning a color having thehighest pixel frequency value as an estimated color to a color shiftarea, thus correcting color shift.

In addition, Japanese Unexamined Patent Application Publication No.9-74750 discloses a color shift correcting apparatus for changing thecombination ratio of an original image and a color shift corrected imagein response to the conditions of the image to form a color shift reducedimage. The apparatus can correct an image derived on undesirableprocessing conditions, for example, upon bleeding or spreading a stain.,thus forming a natural color shift reduced image.

In each of the above-mentioned conventional color shift reducingapparatuses, however, each algorithm is established on the preconditionthat one pigment exists in a living body. Disadvantageously, when bloodcomes out a mucous membrane sprayed with a stain, namely, two or morepigments exist in a living body, the color of the strain is correctedusing the same color as that of the blood, alternatively, the color ofthe blood is corrected using the same color as that of the strain. Whena fixed color is assigned to a color shift portion, the portion iscorrected using an achromatic color.

The present invention is made in consideration of the abovecircumstances. An object of the present invention is to provide a colorshift reducing apparatus capable of accurately estimating a correctioncolor when two or more pigments exist in a living body.

Another object of the present invention is to provide a color shiftreducing apparatus capable of favorably performing a color shiftreducing process to a dark image having low color saturation.

DISCLOSURE OF INVENTION

The present invention provides a color shift reducing apparatusincluding: a color shift area detecting unit for detecting an area withcolor shift in a first image on the basis of a first image signalindicating the first image and a second image signal indicating a secondimage, the first and second images being obtained by imaging a subjectthrough a field-sequential imaging device; a color component signalestimating unit for estimating a color component signal on the basis ofthe other color component signals included in an area of the first imagesignal, the area excluding the color shift area detected through thecolor shift area detecting unit, the other color component signals beingfirst and second color component signals, the color component signalbeing a third color component signal; and a color shift reducing unitfor forming a color shift reduced image on the basis of the first imagesignal and image signals based on the first and second color componentsignals and the third color component signal which is obtained throughthe color component signal estimating unit.

Other features and advantages of the invention will appear more fullyapparent from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 to 10 relate to a first embodiment of the present invention,FIG. 1 being a block diagram of the fundamental structure of a colorshift reducing apparatus, FIG. 2 being a block diagram of the structureof a color shift area detecting unit in FIG. 1, FIG. 3 being a blockdiagram of the structure of a color shift degree calculating unit inFIG. 1, FIG. 4 being a block diagram of the structure of an image dataestimating unit in FIG. 1, FIG. 5 being a block diagram of the structureof a color shift reducing unit in FIG. 1, FIG. 6 being a flowchart ofthe color shift detecting operation performed by the color shift areadetecting unit in FIG. 1, FIG. 7 showing an RGB color space divided into8×8×8 blocks, the space being used in the operation of FIG. 6, FIG. 8being a flowchart of a first process of writing estimated color databased on input image data to LUTs, the process being performed by theimage data estimating unit in FIG. 1, FIG. 9 being a flowchart of asecond process of writing estimated color data based on input image datato the LUTs, the process being performed by the image data estimatingunit in FIG. 1, FIG. 10 showing an example of a color shift degreeconversion graph used in the process of FIG. 9;

FIGS. 11 to 19 relate to a second embodiment of the present invention,FIG. 11 being a block diagram of the structure of a color shift reducingapparatus, FIG. 12 being a block diagram of the structure of a colorshift area detecting unit in FIG. 11, FIG. 13 being a block diagram ofthe structure of a color shift degree calculating unit in FIG. 11, FIG.14 being a block diagram of the structure of an image data estimatingunit in FIG. 11, FIG. 15 being a block diagram of the structure of acolor shift reducing unit in FIG. 11, FIG. 16 being a flowchart of thecolor shift detecting operation performed through the color shift areadetecting unit in FIG. 11, FIG. 17 showing a CrCb color space dividedinto 8×8 areas, the space being used in the operation of FIG. 16, FIG.18 showing a CrCb color space obtained by a dividing method differentfrom that in FIG. 17, FIG. 19 being a flowchart of the operation forwriting estimated color data to LUTs, the operation being performed bythe image data estimating unit in FIG. 11; and

FIGS. 20 and 21 relate to a third embodiment of the present invention,FIG. 20 being a block diagram of the structure of a color shift reducingapparatus, FIG. 21 being a block diagram of the structure of a colorshift area detecting unit in FIG. 20.

BEST MODE FOR CARRYING OUT THE INVENTION

The present invention will now be described in detail with reference tothe appended drawings.

FIRST EMBODIMENT

Referring to FIG. 1, a color shift reducing apparatus 1 comprises: amemory 2 for delaying an input image signal; a color shift areadetecting unit 3 for detecting an area in which color shift occurs onthe basis of input image data and image data delayed through the memory2; a memory 4 for delaying the input image data by one field; a colorshift degree calculating unit 5 for calculating data indicating thedegree of color shift on the basis of a color shift detection signaloutput from the color shift area detecting unit 3, the input image data,and the image data delayed through the memory 4; an image dataestimating unit 6 for estimating image data on the basis of the colorshift detection signal output from the color shift area detecting unit3, the input image data, and the image data delayed through the memory4; and a color shift reducing unit 7 for forming a color shift reducedimage on the basis of the image data output from the memory 4, estimatedimage data output from the image data estimating unit 6, and the colorshift degree data output from the color shift degree calculating unit 5,and then outputting the formed image.

Referring to FIG. 2, the color shift area detecting unit 3 comprises: anRGB-YC matrix circuit 8 for calculating color signals Cr and Cb frominput RGB image data and those of image data delayed through the memory2, and then generating the color signals; a difference detecting circuit9 for calculating a difference between the color signal Cr of the inputimage data and that of the delayed image data and a difference betweenthe color signal Cb of the input image data and that of the delayedimage data to detect a change of a predetermined value or larger on thebasis of the differences; a mean color detecting circuit 10 forcalculating a mean color of the input image on the basis of the colorsignals Cr and Cb of the input image data and then detecting the meancolor; and a color shift determining circuit 11 for determining colorshift on the basis of an output signal of the difference detectingcircuit 9 and that of the mean color detecting circuit 10.

Referring to FIG. 3, the color shift degree calculating unit 5comprises: a histogram forming unit 12 for forming a three-dimensionalhistogram on the basis of the input RGB image data and a color shiftdetection signal generated from the color shift area detecting unit 3; atable forming unit 13 for writing table data for calculation of thedegree of color shift in a look-up table (hereinbelow, referred to as anLUT) 14 with reference to the three-dimensional histogram formed by thehistogram forming unit 12; the LUT 14 for outputting data indicating thedegree of color shift on the basis of image data which is output fromthe memory 4 and is delayed by one field; and a low-pass filter(hereinbelow, referred to as an LPF) 15 for performing a process ofblurring the boundary between a color shift area and an area with nocolor shift to the color shift degree data output from the LUT 14.

Referring to FIG. 4, the image data estimating unit 6 comprises: ahistogram forming unit 16 for forming a three-dimensional histogram fromthe input RGB image data and the color shift detection signal outputfrom the color shift area detecting unit 3; a table forming unit 17 forwriting estimated color data to first to third LUTs 18 a to 18 c withreference to the three-dimensional histogram formed by the histogramforming unit 16; and the first to third LUTs 18 a to 18 c for generatingestimated color data on the basis of the image data delayed by onefield, the image data being generated from the memory 4.

Referring to FIG. 5, the color shift reducing unit 7 comprises: acoefficient determining unit 19 for determining the combination ratio ofthree estimated image data on the basis of the input image data;multiplying units 20 a to 20 c each of which multiplies thecorresponding estimated image data by a coefficient output from thecoefficient determining unit 19; an adding unit 21 for adding datagenerated from the multiplying units 20 a to 20 c; and a combining unit22 for combining correction image data output from the adding unit 21and the image data output from the memory 4 on the basis of the colorshift degree data generated from the color shift degree calculating unit5 to form a color shift reduced image.

The operation of the color shift reducing apparatus constructed asmentioned above will now be described. First, the operation of the colorshift area detecting unit 3 will be described.

Referring to FIG. 6, in step S1, input image data and image data delayedby four fields through the memory 2 are supplied to the RGB-YC matrixcircuit 8, thus calculating the color signals Cr and Cb using thefollowing expressions (1) and (2).Cr=(0.701×R−0.587×G−0.114×B)/1.402   (1)Cb=(−0.299×R−0.587×G+0.886×B)/1.772   (2)

In step S2, the mean color detecting circuit 10 calculates a mean colorof the present one field.

In step S3, the difference detecting circuit 9 calculates |Crn−Crp| and|Cbn−Cbp| from color signals Crn and Cbn of the input image data andcolor signals Crp and Cbp of the image data delayed through the memory4, the respective color signals being obtained by the RGB-YC matrixcircuit 8. In step S4, whether each difference therebetween exceeds apredetermined threshold value is determined. If both the differencestherebetween exceed the threshold value, a determination signal is setto ‘1’. If both the differences therebetween do not exceed the thresholdvalue, the determination signal is set to ‘0’. The determination signalis then supplied to the color shift determining circuit 11 (the polarityof the determination signal can be inverted). If the determinationsignal indicates ‘0’, in step S5, the color shift determining circuit 11determines that there is no color shift and terminates such a process.

If the determination signal indicates ‘1’, in step S6, the color shiftdetermining circuit 11 determines whether the mean color is close to anachromatic color. The reason is as follows: In a case where an image isclose to a monochrome image, if a color produced by color shift is closeto the mean color (achromatic color in this case), the produced colormay stand out. Therefore, when the mean value is close to the achromaticcolor (in other words, the present image is close to a monochromeimage), in the next step S7, the color shift determining circuit 11 doesnot determine whether the produced color is close to the means color. Instep S8, the color shift determining circuit 11 determines that theimage has color shift and then terminates the process.

When the mean color is not close to the achromatic color, in step S7,the color shift determining circuit 11 determines whether the color ofan area, which is currently being subjected to the color shift detectingprocess, is close to the mean color of one field. In other words, whenthe color of the area currently subjected to the color shift detectingprocess is close to the mean color of the field, the color shiftdetermining circuit 11 determines that the area does not include colorshift on the basis of the results obtained by the difference detectingcircuit 9 and the mean color detecting circuit 10. The process proceedsto step S5. It is determined that the image has no color shift. Thus,the process is terminated. When the color of the area currentlysubjected to the color shift detecting process is not close to the meancolor, the color shift determining circuit 11 determines that the areahas color shift. The process proceeds to step S8. It is determined thatthe area has color shift. The process is terminated. The determinationresult is generated as a color shift detection signal.

For example, determination can also be made: A CrCb color plane isdivided into 16×16 areas. When a color, which is currently beingsubjected to the color shift detecting process, corresponds to 3×3 areasincluding and surrounding an area to which the mean color corresponds,it is determined that these areas do not include color shift.

Color shift areas detected using only the difference detecting circuit 9include inconspicuous color shift. Areas which do not need correctionare also corrected. On the other hand, the use of the mean color of onefield calculated through the mean color detecting circuit 10 realizeseffective detection of only visible color shift in the present image.

The above-mentioned process can generate a color shift detection signalobtained by accurately determining a color shift area.

The operation of the image data estimating unit 6 will now be described.

The input image data is supplied to the histogram forming unit 16,together with the color shift detection signal serving as a signalobtained by determining through the color shift area detecting unit 3whether an area has color shift or does not include color shift. Thehistogram forming unit 16 forms a three-dimensional histogram in the RGBcolor space on the basis of image data corresponding to the areadetermined that there is no color shift (steps S11 to S14 in FIG. 8).

According to the present embodiment, it is assumed that the input imagedata comprises eight bits. Therefore, each pixel value of the inputimage data corresponds to any of 256×256×256 points in the RGB colorspace. The RGB color space is divided into 8×8×8 blocks. Incrementing acounter of each block corresponding to image data having no color shiftforms a three-dimensional histogram of one field. FIG. 7 shows the RGBcolor space divided into 8×8×8 blocks.

According to the present embodiment, the RGB color space is divided into8×8×8 areas. The RGB color space can also be divided into anotherdivision number such as 16×16×16 or 32×32×32.

For instance, it is assumed that image data of (R, G, B)=(200, 230, 230)is input and a pixel corresponding to the image data is determined as anarea with no color shift. Higher-order three bits of the input imagedata are referred to. The image data corresponds to a block of (R, G,B)=(6, 7, 7). A counter value of this block is incremented. Theoperation is repetitively performed the number of pixels of one field,thus forming a three-dimensional histogram which is used as a materialto determine which block has many pixels with no color shift.

The table forming unit 17 forms table data to be written to the first tothird LUTs 18 a to 18 c from the formed three-dimensional histogram ofone field.

Referring to steps S15 to S22 of FIG. 9, when forming thethree-dimensional histogram of one field is completed, the table formingunit 17 refers to respective counter values of a group of blocks(hatched portion in FIG. 7) corresponding to (R, G)=(0, 0), namely,eight blocks corresponding to (R, G, B)=(0, 0, 0) to (0, 0, 7). Thetable forming unit 17 determines a value (any of 0 to 7) of B of theblock having the highest frequency value (the number of pixels with nocolor shift is the most) as an estimate of B of the group of blockscorresponding to (R, G)=(0, 0), namely, as estimated color data of B ofeach pixel corresponding to each block in which both R and G indicate 0.Then, the table forming unit 17 writes the estimated color data to thefirst LUT 18 a at an address “000000”. The similar operation issequentially repeated 64 times to process blocks from the group ofblocks corresponding to (R, G)=(0, 0) to a group of blocks correspondingto (R, G)=(7, 7), thus determining estimated color data to be written tothe first LUT 18 a at addresses “000000” to “111111” and writing thedata at the respective addresses.

In the determination of the highest frequency value, when all of thecount values of eight blocks denote zero (the blocks are a group ofblocks having no areas with no color shift in input original imagedata), estimated color data is determined as a predetermined fixed value(zero or a representative value that is often found in endoscopicimages). If there are a plurality of blocks having the highest frequencyvalue, an intermediate value of the blocks (for example, when blockscorresponding to 0 and 7 have the highest frequency value, theintermediate value indicates 2 or 3) or a value of a block that is closeto the block having the second highest frequency value is determined asestimated color data.

Similarly, in steps S23 and 24 and steps S15 to S21 of FIG. 9, theabove-mentioned operation is repetitively performed to groups of blocksof (R, B)=(0, 0) to (7, 7) to determine estimated color data to bewritten to the second LUT 18 b at addresses “000000” to “111111” andwrite the data at the addresses. In steps S23 and S25 and steps S15 toS21 of FIG. 9, the above-mentioned operation is repetitively performedto groups of blocks of (G, B)=(0, 0) to (7, 7), thus determiningestimated color data to be written to the third LUT 18 c at addresses“000000” to “111111” of the third LUT 18 c and writing the data at theaddresses.

The memory 4 delays input image data until writing the estimated colordata formed through the table forming unit 17 on the basis of the imagedata of one field in the first to third LUTs 18 a to 18 c is finished(by about one field). The image data delayed by one field through thememory 4 is input to the first to third LUTs 18 a to 18 c.

The first LUT 18 a uses high-order three bits of each of R and G imagedata values of the image data supplied from the memory 4, assigns thehigh-order three bits of R to high-order three bits of an address in thefirst LUT 18 a, assigns the high-order three bits of G to low-orderthree bits of the address in the first LUT 18 a to read estimated colordata of B from table data written in the first LUT 18 a on the basis ofthe formed address. The read estimated color data of B is combined withdata indicating the high-order three bits of R and data indicating thehigh-order three bits of G, which are used in forming the foregoingaddress in the first LUT 18 a, of the image data supplied from thememory 4. The combined data is output as an estimated image RG.

Similarly, G estimated color data is read out from the second LUT 18 bon the basis of R and B image data value in the image data supplied fromthe memory 4 and combined data is output as an estimated image RB. Restimated color data is read out from the third LUT 18 c on the basis ofG and B image data values and combined data is output as an estimatedimage GB.

The estimated images obtained in the above arrangement have accuracyhigher than that obtained by estimating methods of conventional colorshift reducing apparatuses.

It is assumed that a color of a bleeding portion of (R, G, B)=(200, 20,10) and a color of a stain such as methylene blue of (R, G, B)=(30, 20,150) exist in the same field. This case will now be described (tosimplify the description, it is assumed that other colors do not existin this field).

Pixels corresponding to the color of (R, G, B)=(200, 20, 10) areestimated. If the color is estimated using R and B image data values,the color of (R, G, B)=(200, 20, 10) can be estimated. Conventionally,when the color is estimated using only G image data value, which of thecolor of the bleeding portion and that of the stain is estimated isdetermined depending on the difference between the number of pixels ofan area corresponding to the bleeding portion in the field and thenumber of pixels of a stained area. Accordingly, the bleeding portionmay be estimated using the color of the stain. On the other hand, thestained portion may be estimated using the color of the bleedingportion. When it is assumed that the number of pixels of the stainedarea is larger than that of the area corresponding to the bleedingportion, the color of (R, G, B)=(30, 20, 150) is estimated.

Therefore, an estimated image, obtained by combining data estimated fromthree color components, is influenced by estimated color data based on Gimage data. Unfortunately, the color of the area corresponding to thebleeding portion may be a low-saturation color (achromatic color).

However, according to the estimating method of the color shift reducingapparatus according to the present invention, a color is estimated usingtwo color components of R and G, R and B, or G and B. Accordingly, thecolor of the area corresponding to the bleeding portion of (R, G,B)=(200, 20, 10) can be estimated. Thus, an estimated image obtained bycombining data estimated from three color components can also beestimated with high accuracy.

The operation of the color shift degree calculating unit 5 will now bedescribed.

The histogram forming unit 12 of the color shift degree calculating unit5 operates similar to the histogram forming unit 16 of the image dataestimating unit 6. Therefore, the description of the operation of theunit 12 is omitted.

When the formation of a three-dimensional histogram of one field iscompleted, the table forming unit 13 writes the three-dimensionalhistogram, formed through the histogram forming unit 12, as table datafor calculation of the degree of color shift to the LUT 14. First, acounter value of a block corresponding to (R, G, B)=(0, 0, 0) is readout, the degree of color shift is determined in accordance with theimage size (the number of pixels of one field) of image data, and thedetermined degree is written as table data for calculation of the degreeof color shift to the LUT 14 at an address “000000000”.

According to the present embodiment, the degree of color shift isexpressed using 8-bit data indicating a value in a range of 0 to 255. Asa counter value is larger, the degree of color shift is closer to zero.In other words, it is determined that the corresponding color is notshifted. On the other hand, as a counter value, is smaller, the degreeof color shift is closer to 255, in other words, it is determined thatthe corresponding color is shifted. When it is determined that a coloris not shifted, the corresponding counter value indicates zero. When itis determined that a color is completely shifted, the correspondingcounter value indicates 255. According to the present embodiment, thedegree of color shift is expressed using 8-bit data. The number of bitsof data is not limited to eight bits. Another number of bits such as 16bits or four bits can also be used.

FIG. 10 shows an example of a graph used in converting a counter valueinto the degree of color shift. Which value of 0 to 255 is applied ischanged depending on the image size of image data. For example,referring to FIG. 10, it is assumed that a counter value of image datahaving an image size X is converted into the degree of color shift onthe basis of a conversion graph A. When image data has an image size Ythat is smaller than the image size X, the total number of pixels issmaller than that of the image size X. Therefore, when a counter valueof the image size Y is the same as that of the image size X, thepossibility of color shift is increased. Preferably, the degree of colorshift indicates a value that is higher than that of the image size X.Therefore, in this case, a counter value is converted into the degree ofcolor shift on the basis of a conversion graph B.

Similar to the case of (R, G, B)=(0, 0, 0), counter values of 512 blockscorresponding to (R, G, B)=(0, 0, 1) to (7, 7, 7) are read out in total.The degrees of color shift are written to the LUT 14 at addresses“000000000” to “111111111” in each of which a value indicating R isassigned to the most significant bit, a value indicating G is assignedto the next high-order bit, and a value indicating B is assigned to theleast significant bit, thus completely forming a table for calculationof the degree of color shift in the LUT 14.

Image data delayed through the memory 4 is supplied to the LUT 14completed as the table for calculation of the degree of color shift. Thedegree of color shift of the corresponding block is read from the RGBcolor space every pixel. The degree of color shift is output to the LPF15, thus smoothing the degree of color shift in accordance with thedegrees of color shift of surrounding pixels.

The LPF 15 comprises a 3×3 smoothing filter. In forming a color shiftreduced image, the LPF 15 suppresses a variation or a steep change indegrees of color shift between adjacent pixels so that an unnaturalimage is not produced. According to the present embodiment, the size of3×3 is used. The other size such as 5×5 can also be used. The size canbe changed in accordance with conditions of an image.

The operation of the color shift reducing unit 7 will now be described.

In endoscopic images, a change in color is dominated by blood or a stain(such as methylene blue). In the images, in many cases, the brightnessdistribution of only one color component of R, G, and B color componentsis wide (for 8-bit data, brightness values are distributed in a widerange of 0 to 255) and brightness values of the other color componentsare concentrated to low values (for example, when a change in color isdominated by blood, the only one color component is R, or when a changein color is dominated by the stain, the only color component is B).

In the estimation of data indicating the other color components on thebasis of color component data of the above image using the methodaccording to the present embodiment, color component data having narrowbrightness distribution is concentrated to limited blocks in the RGBcolor space of FIG. 7. Accordingly, estimated colors are restricted tosome colors.

Therefore, the following tendency is shown: An estimated image derivedfrom color component data with wide brightness distribution has higheraccuracy than an estimated image derived from color component data withnarrow brightness distribution.

In endoscopic images, the respective mean values of the R, G, and Bcolor components of the images are calculated. The mean value of a colorwith wide brightness distribution tends to be high and that of a colorwith narrow brightness distribution tends to be low. When both of bloodand a stain (methylene blue) govern a change in color in substantiallythe same proportion, both of R and B color components have widebrightness distribution. The mean value of each color component tends tobe intermediate (about 128 in the case of 8-bit data). The G colorcomponent has narrow brightness distribution and the mean value thereoftends to be low. In this case as well, when brightness distribution of acolor component is wide, the mean value thereof tends to be high. Thepresent embodiment uses the above-mentioned properties. Advantageously,the scale of a circuit can be reduced as much as possible. Accordingly,the combination ratio simply reflecting the above tendency ofdistribution of color components is calculated using the mean values.

The coefficient determining unit 19 calculates the combination ratio ofthe estimated image RG from R and G data, that of the estimated image RBfrom R and B data, and that of the estimated image GB from G and B imagedata on the basis of the mean values of the R, G, and B color componentsof the input image data.

First, mean values {overscore (R_(AVG))}, {overscore (G_(AVG))} and{overscore (B_(AVG))} of one field of R, G, and B of the input imagedata are calculated. A mean value of {overscore (R_(AVG))} and{overscore (G_(AVG))}, that of {overscore (R_(AVG))} and {overscore(B_(AVG))}, and that of {overscore (G_(AVG))} and {overscore (B_(AVG))}are calculated from the obtained mean values {overscore (R_(AVG))},{overscore (G_(AVG))}, and {overscore (B_(AVG))}. Let {overscore(RG_(AVG))}, {overscore (RB_(AVG))}, and {overscore (GB_(AVG))} be theobtained mean values, respectively.

Let Crg, Crb, and Cgb denote the combination ratio of the estimatedimage RG, that of the estimated image RB, and that of the estimatedimage GB, respectively. The combination ratios of the respectiveestimated images are calculated using the following expressions (3) to(5). The ratios Crg, Crb, and Cgb are output to the multiplying units 20a, 20 b, and 20 c, respectively.Crg={overscore (RG _(AVG) )}/({overscore ( RG _(AVG))}+{overscore (RB_(AVG) )} +{overscore (GB _(AVG) )})   (3)Crb={overscore (RB _(AVG) )}/({overscore ( RG _(AVG))}+{overscore (RB_(AVG) )} +{overscore (GB _(AVG) )})   (4)Cgb={overscore (GB _(AVG) )}/({overscore ( RG _(AVG))}+{overscore (RB_(AVG) )} +{overscore (GB _(AVG) )})   (5)

The multiplying unit 20 a multiplies the estimated image data RG outputfrom the first LUT 18 a by a coefficient Crg generated from thecoefficient determining unit 19 and then generates obtained data to theadding unit 21.

Similarly, the multiplying unit 20 b multiplies the estimated image dataRB by a coefficient Crb and the multiplying unit 20 c multiplies theestimated image data GB by a coefficient Cgb. Each obtained data issupplied to the adding unit 21.

The adding unit 21 adds the data supplied from the multiplying units 20a to 20 c to form correction image data.

The correction image data formed as mentioned above is calculated everyfield at the optimum combination ratio based on the tone of the wholeframe. The correction image data is formed with higher accuracy thanthat of the conventional color shift reducing apparatus.

The combining unit 22 combines the correction image data output from theadding unit 21 and the image data supplied from the memory 4 on thebasis of the color shift degree data generated from the color shiftdegree calculating unit 5. As the possibility of color shift is higher,the color shift degree data is closer to 255. Accordingly, when thecolor shift degree data indicates 255, 100% of the correction image datais used in combination. On the other hand, when the color shift degreedata indicates zero, 100% of the image data supplied from the memory 4is used in the combination. When the color shift degree data indicates128, the correction image data is combined with the image data suppliedfrom the memory 4 in the ratio of 5:5.

As mentioned above, according to the color shift reducing apparatusaccording to the present embodiment, the accuracy of an estimated imageto be applied to a color shift area is higher than that of theconventional color shift reducing apparatus. Consequently, a color shiftreduced image to be applied to an image in which both blood and a stainexist in the same field is obtained as a natural image. Thus, a naturalcolor shift reduced image can be formed.

SECOND EMBODIMENT

Referring to FIG. 11, the structure of a color shift reducing apparatus23 according to the present embodiment is substantially the same as thatof the color shift reducing apparatus according to the first embodimentexcluding the supply of a color shift reduced image to a color shiftarea detecting unit 24, the internal structure of a color shift degreecalculating unit 25, the internal structure of an image data estimatingunit 26, and the internal structure of a color shift reducing unit 27.Accordingly, the difference therebetween will now be described. The samecomponents are designated by the same reference numerals to omit thedescription of the same components.

Referring to FIG. 12, the color shift area detecting unit 24 comprises:an RGB matrix circuit 28 for calculating color signals Cr and Cb and abrightness signal Y from input image data of RGB and a color shiftreduced image processed through the color shift reducing unit 27 andthen outputting the calculated signals; and a difference detectingcircuit 29 for calculating a difference between the signal Cr of theinput image data and that of the color shift reduced image processedthrough the color shift reducing unit 27 and a difference between thesignal Cb of the input image data and that of the color shift reducedimage processed through the color shift reducing unit 27 and thendetermining whether each difference therebetween changes by apredetermined value or more, the predetermined value being based oninformation of the brightness signal Y of the color shift reduced image.The other components are the same as those according to the firstembodiment.

Referring to FIG. 13, the color shift degree calculating unit 25comprises: an RGB-YC matrix circuit 30 for converting each of inputimage data of RGB and image data delayed by one field, the image databeing supplied from the memory 4, into color signals Cr and Cb; ahistogram forming unit 31 for forming a two-dimensional histogram on thebasis of the color signals Cr and Cb converted from the input RGB imagedata and a color shift detection signal output from the color shift areadetecting unit 24; a table forming unit 32 for writing table data forcalculation of the degree of color shift to an LUT 33 with reference tothe two-dimensional histogram formed by the histogram forming unit 31;and the LUT 33 for generating data indicating the degree of color shifton the basis of the color signals Cr and Cb which are output from theRGB-YC matrix circuit 30 and are obtained by converting the RGB imagedata delayed through the memory 4. The other component is the same asthat of the first embodiment.

Referring to FIG. 14, the image data estimating unit 26 comprises: atable forming unit 34 for writing estimated color data to LUTs 35 a to35 c with reference to the three-dimensional histogram formed throughthe histogram forming unit 16; and the first to third LUTs 35 a to 35 cfor generating estimated color data on the basis of the image dataoutput from the memory 4, the image data being delayed by one field. Theother component is the same as that of the first embodiment.

Referring to FIG. 15, the color shift reducing unit 27 comprises acombining unit 36 for combining correction image data output from theadding unit 21 and the image data output from the memory 4 on the basisof the color shift degree data output from the color shift degreecalculating unit 25 to form a color shift reduced image. The othercomponents are the same as those of the first embodiment.

The operation of the color shift reducing apparatus constructed asmentioned above will now be described.

The operation of the color shift area detecting unit 24 will now bedescribed with reference to FIG. 16.

Referring to FIG. 16, the input image data and the color shift reducedimage data output from the color shift reducing unit 27 are supplied tothe RGB-YC matrix circuit 28. In a manner similar to the firstembodiment, the color signals Cr and Cb are calculated in step S51(refer to the expressions (1) and (2)). In addition, the brightnesssignal Y is calculated from color shift reduced image data by thefollowing expression (6).Y=0.3×R+0.59×G+0.11×B   (6)

In step S52, the mean color detecting circuit 10 calculates a mean colorof the present one field.

In the same way as the first embodiment, in step S53, the differencedetecting circuit 29 calculates |Crn−Crp|and |Cbn−Cbp| on the basis ofcolor signals Crn and Cbn of the input image data and color signals Crpand Cbp of the color shift reduced image data, the color signals beingobtained by the RGB-YC matrix circuit 28. Whether each of |Crn−Crp| and|Cbn−Cbp| exceeds a predetermined threshold value is determined. Thesecond embodiment differs from the first embodiment in that thethreshold value varies depending on the mean brightness value YAVG ofthe brightness signal Y of the color shift reduced image data.

A slight change in color, namely, slight color shift may stand upconspicuously in a dark image with low saturation. In this image, colorsignals Cr and Cb of the whole image tend to be low (achromatic andmonochrome image). In color shift determination using a fixed thresholdvalue in the same way as the first embodiment, slight color shift maynot be detected. Therefore, when the mean brightness value {overscore(Y_(AVG))} of the color shift reduced image data is low, it is necessaryto set the threshold value to a low value. In step S54, the thresholdvalue is set on the basis of the mean brightness value {overscore(Y_(AVG))} of the color shift reduced image data.

As mentioned above, the threshold value used in color shift detection isvaried in accordance with the brightness of the image and a result ofthe color shift detection is output to the color shift determiningcircuit 11. The operation and function of the color shift determiningcircuit 11 are the same as those of the first embodiment. Steps S4 to S8described in FIG. 6 are executed.

The color shift detection performed in the above structure can improvethe detection of color shift in a dark image, the color shift in thedark image being difficult to detect according to the first embodiment.

According to the present embodiment, the mean brightness value of thecolor shift reduced image data is obtained and used in setting thethreshold value. The mean brightness value of input image data can alsobe obtained and used in setting the threshold value.

The operation of the color shift degree calculating unit 25 will now bedescribed.

The input image data and the image data, supplied from the memory 4 anddelayed by one field, are input to the RGB-YC matrix circuit 30 tocalculate the color signals Cr and Cb of the respective image data. Thecolor signals of the input image data are output to the histogramforming unit 31. The color signals of the image data supplied from thememory 4 are output to the LUT 33.

The color signals of the input image data are supplied to the histogramforming unit 31, together with a color shift detection signal generatedthrough the color shift area detecting unit 24. The histogram formingunit 31 forms a two-dimensional histogram in a CrCb color space on thebasis of image data which is determined as image data with no colorshift.

According to the present embodiment, input image data comprises 8 bitsin the same way as the first embodiment. Therefore, each pixel value ofinput image data corresponds to any of 256×256 points in the CrCb colorspace. The CrCb color space is divided into 8×8 blocks. A counter ofeach block to which image data with no color shift corresponds isincremented to form a two-dimensional histogram of one field. FIG. 17shows the CrCb color space divided into 8×8 blocks.

The present embodiment uses the division shown in FIG. 17. Anothernumber of blocks such as 16×16 can also be used. Dividing the colorspace into blocks using the origin at the center can also be used asshown in FIG. 18.

When the formation of the two-dimensional histogram of one frame isfinished, the table forming unit 32 obtains table data for calculationof the degree of color shift from the two-dimensional histogram formedthrough the histogram forming unit 31, and then writes the obtained datato the LUT 33.

First, a counter value of a block corresponding to (Cr, Cb)=(0, 0) isread out. The degree of color shift is determined in accordance with theimage size (the number of pixels of one field) of image data. Thedetermined degree is written as table data for calculation of the degreeof color shift to the LUT 33 at an address “000000”.

Since the method for converting a counter value of each block into thedegree of color shift is the same as that of the first embodiment, thedescription regarding the method is omitted.

In a manner similar to the first embodiment, counter values of 64 blockscorresponding to (Cr, Cb)=(0, 0) to (7, 7) in total are read out. Thedegree of color shift is written to the LUT 33 at an address “000000” inwhich a value of Cr is assigned to the most significant bit and a valueof Cb is assigned to the least significant bit, thus completely formingthe LUT 33 as a color shift degree calculation table.

The color signals of the image data supplied from the memory 4 are inputto the LUT 33 functioning as the color shift degree calculation table,the color signals being obtained by converting the image data throughthe RGB-YC matrix circuit 30. The degree of color shift of thecorresponding block in the CrCb color space is read every pixel. Theread degrees are output to the LPF 15 to smooth the degree of colorshift in the same way as the first embodiment.

Since the operation and function of the LPF 15 are the same as those ofthe LPF 15 in the first embodiment, the description thereof is omitted.

The operation of the image data estimating unit 26 will now bedescribed.

Since the operation and function of the histogram forming unit 16 arethe same as those of the unit 16 in the first embodiment, thedescription thereof is omitted (refer to FIG. 8).

The table forming unit 34 forms table data from the formedthree-dimensional histogram of one field in steps S115 to 126 of FIG. 19and writes the table data to the first to third LUTs 35 a to 35 c. Whenthe formation of the three-dimensional histogram of one field iscompleted, the table forming unit 34 refers to counter values of eightblocks included in a group of blocks corresponding to (R, G)=(0, 0) todetermine a value of B of the block having the highest frequency valuein the same way as the first embodiment.

The present embodiment differs from the first embodiment in estimatedcolor data written in the first LUT 35 a. According to the presentembodiment, the table forming unit 34 calculates Cr and Cb from (R, G,B)=(0, 0) (value of the block having the highest frequency value)obtained as mentioned above and then writes Cr and Cb as estimated colordata to the first LUT 35 a at the address “000000”. The similaroperation is sequentially performed to block groups corresponding to (R,G)=(0, 0) to (7, 7), namely, the operation is repeated 64 times todetermine estimated color data to be written to the first LUT 35 a atthe addresses “000000” to “111111” and write the data.

In this instance, when the counter values of eight blocks, used indetermining the highest frequency value, indicate zero, mean colorsignals {overscore (Cr_(AVG))} and {overscore (Cb_(AVG))} of originalinput image data are obtained. The values of the signals are determinedas estimated color data. When there are a plurality of blocks having thehighest frequency value, the operation similar to that of the firstembodiment is performed. Accordingly, the description of the similaroperation is omitted.

In each of the first to third LUTs 35 a to 35 c, in the same way as thefirst embodiment, an address is formed to read out the estimated colordata Cr and Cb. The read data is output as an estimated image to thecolor shift reducing unit 26.

The operation of the color shift reducing unit 26 will now be described.

The coefficient determining unit 19, the multiplying units 20, and theadding unit 21 differ from those of the first embodiment in that Cr andCb are used as data to be operated instead of R, G, and B. However, theoperations of these units are the same as those of the first embodiment.Accordingly, the description thereof is omitted.

The combining unit 36 combines correction image data output from theadding unit 21 and the image data output from the memory 4 on the basisof the color shift degree data generated from the color shift degreecalculating unit 25.

Since the correction image data output from the adding unit 21 includecolor signals Cr and Cb, the color signals Cr and Cb are calculated fromthe RGB image data output from the memory 4 and the respective colorsignals are combined on the basis of the color shift degree data. Sincethe combining method is the same as that in the first embodiment, thedescription thereof is omitted. The combined color signals Cr and Cb areconverted together with the brightness signal Y output from the RGBimage data generated from the memory 4 into RGB image data using thefollowing expressions (7) to (9).R=Y+1.402×Cr   (7)G=Y−0.714×Cr−0.344×Cb   (8)B=Y+1.772×Cb   (9)

The color shift reducing apparatus constructed as mentioned above canperform a color shift reducing process to a dark image having lowsaturation with higher accuracy than the color shift reducing apparatusaccording to the first embodiment.

THIRD EMBODIMENT

Referring to FIG. 20, a color shift reducing apparatus 37 according tothe present embodiment comprises: a memory 38 for delaying an inputimage by one field; and a color shift area detecting unit 39 fordetecting color shift on the basis of the image data delayed by onefield, the data being supplied from the memory 38, the input image dataof the color shift reducing apparatus 37, and a color shiftreduced-image output from the color shift reducing unit 27. The othercomponents are the same as those in the color shift reducing apparatusaccording to the second embodiment. A difference between theconstruction of the apparatus according to the third embodiment and thataccording to the second embodiment will now be described. The samecomponents are designated by the same reference numerals to omit thedescription thereof.

Referring to FIG. 21, the color shift area detecting unit 38 comprisesan RGB-YC matrix circuit 40 for calculating color signals Cr and Cb anda brightness signal Y from the input RGB image data, the image datadelayed through the memory 38, and the color shift reduced image dataobtained through the color shift reducing unit 27. Since the othercomponents are the same as those in the second embodiment, thedescription thereof is omitted.

The operation of the color shift reducing apparatus constructed asmentioned above will now be described.

The image data supplied to the color shift reducing apparatus is delayedby one field through the memory 38 and is then output to the color shiftarea detecting unit 39. In the RGB-YC matrix circuit 40 of the colorshift area detecting unit 39, in the same way as the second embodiment,the color signals Cr and Cb of the image data supplied to the colorshift reducing apparatus and color signals Crp and Cbp of the colorshift reduced image generated from the color shift reducing unit 27 arecalculated. In addition, color signals Crn and Cbn of the image dataoutput from the memory 38 and the brightness signal Y of the input imagedata are calculated.

The color signals Cr and Cb of the image data supplied to the colorshift reducing apparatus are output to the mean color detecting circuit10. The other color signals Crn, Cbn, Crp, and Cbp and the brightnesssignal Y are output to the difference detecting circuit 29. The signalsare processed in the same way as the second embodiment.

The operations and functions of the other components are the same asthose of the second embodiment. Accordingly, the description thereof isomitted.

According to the second embodiment, the timing of a mean value{overscore (Y_(AVG))} of the mean color or the brightness signal isdelayed by one field. According to the present embodiment, this timingmatches the timing of an image which is actually processed, thusproviding a color shift reduced image having advantages over those ofthe second embodiment.

According to the present embodiment, the image data supplied to thecolor shift reducing apparatus is input to the memory 38 and is thendelayed by one field. Supplying and delaying the color shift reducedimage output from the color shift reducing unit 27 provides similaradvantages.

In the present invention, it is apparent that different embodiments in awide range can be formed on the basis of this invention withoutdeparting from the spirit and scope of the invention. This invention isrestricted to the appended claims but not be limited to any particularembodiment.

Industrial Applicability

As mentioned above, the color shift reducing apparatus according to thepresent invention is effectively used to estimate a correction colorwith high accuracy when two or more pigments exist in a living body.

1. A color shift reducing apparatus comprising: a color shift area detecting unit for detecting an area with color shift in a first image on the basis of a first image signal indicating the first image and a second image signal indicating a second image, the first and second images being obtained by imaging a subject through a field-sequential imaging device; a color component signal estimating unit for estimating a color component signal on the basis of the other color component signals included in an area of the first image signal, the area excluding the color shift area detected through the color shift area detecting unit, the other color component signals being first and second color component signals, the color component signal being a third color component signal; and a color shift reducing unit for forming a color shift reduced image on the basis of the first image signal and image signals based on the first and second color component signals and the third color component signal which is obtained through the color component signal estimating unit.
 2. The color shift reducing apparatus according to claim 1, wherein the color shift reducing unit comprises: three estimated-color-image forming units for forming estimated color images on the basis of the three color component signals obtained through the color component signal estimating unit, respectively; a color shift corrected image forming unit for combining the three estimated color images output from the three estimated-color-image forming units to form a color shift corrected image; and a color shift reduced image forming unit for combining the color shift corrected image formed through the color shift corrected image forming unit and the first image signal to form a color shift reduced image.
 3. The color shift reducing apparatus according to claim 2, wherein the color shift corrected image forming unit determines the combination ratio of the three estimated color images on the basis of information related to distribution of the respective color component signals of the first image signal.
 4. The color shift reducing apparatus according to claim 1, wherein when it is impossible to estimate one color component signal on the basis of the other two color component signals of the three color component signals in the area excluding the color shift area detected through the color shift area detecting unit, the color component signal estimating unit estimates the one color component signal on the basis of a mean value of data of one field in the first image signal.
 5. A color shift reducing apparatus comprising: color shift area detecting means for detecting an area with color shift in a first image on the basis of a first image signal indicating the first image and a second image signal indicating a second image, the first and second images being obtained by imaging a subject through a field-sequential imaging device; color component signal estimating means for estimating a color component signal on the basis of the other color component signals included in an area of the first image signal, the area excluding the color shift area detected through the color shift area detecting means, the other color component signals being first and second color component signals, the color component signal being a third color component signal; and color shift reducing means for forming a color shift reduced image on the basis of the first image signal and image signals based on the first and second color component signals and the third color component signal, which is obtained through the color component signal estimating means. 